Paper Classification Based on Three-Dimensional Characteristics

ABSTRACT

Examples disclosed herein relate to classifying paper based on three-dimensional characteristics of the paper. For example, a representation of the three-dimensional characteristics of the paper may be created, and statistical summary information related to the three-dimensional characteristics of the paper may be determined based on the representation. The paper may be classified based on the statistical summary information.

BACKGROUND

Printing may be performed on different types of paper substrates, suchas labels, traditional paper, photographic paper, or card stock. Thepapers may have different topography from one another. For example, cardstock with a glossy appearance may have a different surface texture thancard stock with a more matte finish.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings describe example embodiments. The following detaileddescription references the drawings, wherein:

FIG. 1 is a block diagram illustrating one example of a computing systemto classify paper based on three-dimensional characteristics.

FIGS. 2A and 2B are diagrams illustrating one example of an imagingsystem to capture an image of paper to classify based onthree-dimensional characteristics.

FIG. 2 is a flow chart illustrating one example of a method to classifypaper based on three-dimensional characteristics.

FIG. 3 is a diagram illustrating one example of classifying paper basedon three-dimensional characteristics.

DETAILED DESCRIPTION

In one implementation, a paper is automatically classified according tothree-dimensional characteristics of the paper. The classification maybe performed based on a photometric stereo mapping of the paper createdfrom images of the paper illuminated from different light sources. Athree-dimensional representation of the paper may be determined based onthe photometric stereo mapping of the paper. Summary information aboutthe three-dimensional characteristics of the paper may be determinedsuch that the paper may be more easily compared to classificationinformation. For example, a machine-learning method may be performed tocreate a model for classifying paper, and the summary information may beclassified according to the model.

Small topographical differences in paper may affect the way printedmaterial appears on the paper. As such, it may be desirable to changethe way content is printed based on the classification of the papersubstrate. For example, color processing may be performed differentlybased on the paper classification. The amount of toner and/or ink may beadjusted according to the paper classification to alter the colorprocessing.

FIG. 1 is a block diagram illustrating one example of a computing systemto classify paper based on three-dimensional characteristics. Forexample, a paper printing substrate may be classified according to itstopography, and print parameters may be selected based on theclassification. The computing system 100 includes a processor 101, amachine-readable storage medium 102, a storage 103, and a photometricstereo imaging system 104.

The photometric stereo imaging system 104 may be used to estimatesurface normals of paper based on images of the paper in differentlighting conditions. The photometric stereo imaging system 104 may useany suitable camera. In one implementation, a Dyson relay lens with 1:1magnification is used. It may use a fixed focal length and fixed focuscontact device. The photometric stereo imaging system 104 may include anair gap between the imaging device and the target paper, such as a 4.5mm gap. The gap may allow for more options for the illumination sourcepositioning.

Illumination from different sources may prevent a directional componentfrom a single light source giving a directional filtering effect. Insome implementations, at least three light sources are used. Forexample, the photometric stereo imaging system 104 may include fourlight sources. The light sources may be, for example, light emittingdiodes (LEDs). A light source may be placed at each of the corners of aninput aperture of a relay lens associated with the photometric stereoimaging system 104. The light sources may be controlled independentlysuch that they may illuminate together and/or some of the light sourcesmay not be used. For example, a serial interface may be used to controlthe light sources. Multiple images of the target paper may be capturedwith three or more known illumination surfaces, and the images may beused to create a mapping of the surface slant and albedo of the targetpaper. The input images of the target paper used to create the mappingmay be, for example, high dynamic range (HDR) images. As an example, thephotometric stereo technique may be performed as disclosed by R. J.Woodham in “Photometric Method for Determining the Surface Orientationfrom Multiple Images” published in Optical Engineering, Volume 19 No. 1in 1980.

FIGS. 2A and 2B are diagrams illustrating one example of an imagingsystem to capture an image of paper to classify based onthree-dimensional characteristics. For example, Dyson relay lens 200includes an additional external aperture 201. The aperture 201 includes4 light sources 202, 203, 204, and 205. The Dyson relay lens 200captures an image of paper 206 and has an air gap 207 from the aperture201 and the paper 206. The air gap 207 may be, for example, 4.5 mm.

Referring back to FIG. 1, the storage 103 may be a storage for storinginformation accessible to the processor 101. For example, the storage103 and processor 101 may be included in the same electronic device ormay communicate via a network. The storage 103 may include paperclassification information 105. The paper classification information 105may relate to three-dimensional characteristics of paper. In oneimplementation, the paper classification information 105 is determinedby the processor 101 or another processor. The paper classificationinformation 105 may be determined based on a machine learning modelapplied to paper with known classifications. In one implementation, thepaper classification information 105 is a neural network model forclassifying paper printing substrates.

The processor 101 may be a central processing unit (CPU), asemiconductor-based microprocessor, or any other device suitable forretrieval and execution of instructions. As an alternative or inaddition to fetching, decoding, and executing instructions, theprocessor 101 may include one or more integrated circuits (ICs) or otherelectronic circuits that comprise a plurality of electronic componentsfor performing the functionality described below. The functionalitydescribed below may be performed by multiple processors.

The processor 101 may communicate with the machine-readable storagemedium 102. The machine-readable storage medium 102 may be any suitablemachine readable medium, such as an electronic, magnetic, optical, orother physical storage device that stores executable instructions orother data (e.g., a hard disk drive, random access memory, flash memory,etc.). The machine-readable storage medium 102 may be, for example, acomputer readable non-transitory medium. The machine-readable storagemedium 102 may include instructions executable by the processor 101. Forexample, the machine-readable storage medium 102 includes paperthree-dimensional description instructions 106, three-dimensionalsummary statistics determination instructions 107, and paperclassification instructions 108.

The paper three-dimensional description instructions 106 may includeinstructions to create a description of the three-dimensionalcharacteristics of a paper. The paper may be paper, such as cardstock orlabels, used as a printing substrate. A surface mapping of thethree-dimensional characteristics of the paper may be determined from asurface normal map created from the photometric stereo imaging system104. The description of the three-dimensional characteristics may be,for example, a shape index representing the three-dimensional shape ofthe paper at different locations along the paper. As an example, a shapeindex for indicating the topography of an object at different portionsmay be similar to the shape index described in “Surface Shape andCurvature Scales” published by J. J. and van Doom and A. J. Koenderinkin Image and Vision Computing, Vol. 10 No. 8 in 1992.

The three-dimensional summary statistics determination instructions 107may include instructions to summarize the description of thethree-dimensional characteristics. For example, a more concise versionof the description of the three-dimensional characteristics may becreated to be used to classify the paper. The summary statistics mayinclude, for example, contrast, energy, and/or entropy informationrelated to the three-dimensional characteristics.

The paper classification instructions 108 may include instructions toclassify the paper based on a comparison of the summary statistics ofthe paper to the paper classifying information 105. In some cases, thecomparison may result in the paper being determined to be an unknowntype, such as where the three-dimensional characteristics of the papersubstrate do not resemble those of the categories.

In one implementation, the machine-readable storage medium 102 isassociated with a printer, and instructions for classifying a paperprinting substrate are executed when a paper is provided to the printer.The paper may be classified, and printer settings may be automaticallydetermined based on the paper printing substrate.

FIG. 3 is a flow chart illustrating one example of a method to classifypaper based on three-dimensional characteristics. For example, theclassification may be based on photometric stereo information related tothe paper. The paper may be classified based on a machine-learning modelcreated from paper with known classifications, and the paperclassification information may be used to adjust printer settingsaccording to the topography of the paper. The method may be implemented,for example, by the processor 101 from FIG. 1.

Beginning at 300, a processor creates a representation of thethree-dimensional characteristics of a paper based on photometric stereoinformation related to the paper. The paper may be, for example, a card,label, photographic paper, sheet of paper, box, or other paper substratefor printing. In one implementation, multiple images are captured usingdifferent illumination sources. For example, the paper may beilluminated using at least three light emitting diodes. In oneimplementation, a rectangular image of a portion of the paper iscaptured. Four light emitting diodes may be used such that a lightsource is associated with each of the four corners of the image.

In one implementation, the three-dimensional representation is createdby analyzing curvature information related to the photometric stereodata. This may be performed, for example, by determining first orderderivatives along the x-axis and y-axis, determining the mean of thefirst order derivatives, and determining principle curves from the meancurvature information. The principle curve information may be used todetermine a shape index for the paper surface. The shape index mayrepresent a data of curvature information of the paper surface. Forexample, the shape index may include an entry for representing thethree-dimensional characteristic of each pixel or other portion of ashape map of the paper. In one implementation, the shape index S isformed by

$S = {\frac{2}{\pi}{arc}\; {\tan \left( \frac{k_{2} + k_{1}}{k_{2} - k_{1}} \right)}}$

Where k₁ and k₂ represent principal curvatures measured directly fromthe surface normals. The surface normals may be obtained from thephotometric stereo data. The three-dimensional representation mayinclude information about both the paper and ink or other items on thepaper.

Continuing to 301, a processor determines a statistical signature of thethree-dimensional characteristics of the paper based on therepresentation. The signature may provide a reduced amount of data thatcan be analyzed to classify the paper. The signature may be created bysummarizing any relevant three-dimensional information associating withthe paper. The information may include, for example, contrast, energy,and/or entropy of the paper. In one implementation, the signature isderived from a co-occurrence matrix related to relative changes inthree-dimensional characteristics across the paper.

In one implementation, co-occurrence information may be extracted from ashape index map of the paper. The co-occurrence information may bedetermined in a horizontal and vertical direction for changes inpatches, such as a pixel or set of pixels, of the shape index map. Forexample, the three-dimensional signature may include horizontalcontrast, vertical contrast, horizontal energy, vertical energy, andentropy. A co-occurrence matrix may be derived from the shape indexusing a vertical or horizontal offset. Summarization contrast and energyvalues may be determined based on the co-occurrence matrix, and entropymay be measured directly from the shape index map.

Moving to 302, a processor classifies the paper based on the signature.In one implementation, the paper is classified according to a machinelearning model. For example, a machine learning model, such as a neuralnetwork, may be created by classifying papers with knownclassifications. For example, multiple images using differentillumination sources may be taken of a paper with a known type. Theimages may be taken of different locations of the paper and thedifferent locations of the paper may be associated with differentsurface types. A portion of the sample papers may be used for training aneural network or other machine learning model, a portion forvalidation, and a portion for testing. The training features mayinclude, for example, energy, contrast, and entropy. In some cases, theclassification may indicate that the paper is in an uncategorized group,such as where the topography of the paper does not relate to any of thelearned categories.

In one implementation, printing parameters are selected based on thethree-dimensional characteristics of the paper. For example, color mayappear differently on paper according to the topography of the paper. Toaccount for differences in the texture of paper printing substrates, anInternational Color Code (ICC) profile may be associated with each ofthe paper categories. When the paper is categorized, the associatedprofile is used by the printer for printing on the paper.

In one implementation, the classification is used to identify the sourceof the paper and/or ink on the paper. For example, the differentclassifications may represent different sources. Classifying the papermay be used to confirm that the paper is associated with the expectedsource. If the paper is not associated with the expected source,counterfeiting may be expected. For example, the processor may outputinformation indicating a level of likelihood of counterfeiting.

FIG. 4 is a diagram illustrating one example of classifying paper basedon three-dimensional characteristics. At 400, paper is selected to beanalyzed, for example, by a user inserting the paper into a printer. At401, a processor creates a photometric stereo map of the paper. Thephotometric stereo map may be created from multiple images of the paperwhere each image is illuminated with a different position of a lightsource. At 402, information from the photometric stereo map is used tocreate a representation of the three-dimensional characteristics of thepaper. For example, the representation may be a shape index map. At 403,the representation of three-dimensional characteristic information isused to summarize the three-dimensional characteristics of the paper. At304, classification model information 304 and three-dimensionalcharacteristics summary information 403 is used to classify the paper.The classification may be used, for example, to automatically adjustprinter settings.

1. A computing system, comprising: a photometric stereo imaging systemwith three light sources to capture multiple images of a paper; astorage to store paper classification information related tothree-dimensional statistics of paper types; and a processor to:determine a description of the three-dimensional characteristics of apaper based on images of the paper captured by the photometric stereoimaging system; determine statistical summary information related to thethree-dimensional characteristics description; and classify the paperbased on a comparison of the statistical summary information to thestored paper classification information.
 2. The computing system ofclaim 1, wherein the processor further: determines printing parametersbased on the paper classification; and causes a printer to print on thepaper using the determined print parameters.
 3. The computing system ofclaim 1, wherein the light source comprises four illumination sourcesand wherein a light source is associated with each of the corners of animage of the paper.
 4. The computing system of claim 1, whereinclassifying the paper comprises associating the paper with a category ofunknown paper type.
 5. The computing system of claim 1, wherein thestored paper classification information is determined based on a machinelearning model applied to paper with known classifications.
 6. A method,comprising: creating, by a processor, a representation ofthree-dimensional characteristics of a paper based on photometric stereoinformation related to the paper, determining a statistical signature ofthe three-dimensional characteristics of the paper based on therepresentation; and classify the paper based on the signature.
 7. Themethod of claim 6, wherein creating a representation comprises creatinga shape index indicating three-dimensional shape characteristics of thepaper.
 8. The method of claim 6, wherein determining the signaturecomprises summarizing three-dimensional information related to at leastone of contrast, energy, and entropy.
 9. The method of claim 6, whereindetermining the signature comprises creating a co-occurrence matrixrelated to relative changes in three-dimensional characteristics acrossthe paper.
 10. The method of claim 6, further comprising selectingprinting parameters based on the classification.
 11. The method of claim6, further comprising performing a machine learning method to create amodel to classify paper types based on three-dimensional signatures ofpapers with known classifications.
 12. The method of claim 11, whereinclassifying the paper comprises classifying the paper according to themodel.
 13. The method of claim 11, wherein the machine learning methodcomprises a neural network.
 14. A machine-readable non-transitorystorage medium comprising instructions executable by a processor to:create a representation of three-dimensional characteristics of a paperbased on photometric stereo data related to the paper; determinestatistical summary information related to the three-dimensionalcharacteristics of the paper based on the representation; and classifythe paper based on the statistical summary information.
 15. Themachine-readable non-transitory storage medium of claim 14, furthercomprising instructions to select print parameters based on theclassification.